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LEXINGTON 


MASSACHUSETTS 


ABSTRACT 


The  capabilities  of  the  maximum  likelihood  estimator  (MLE) 
and  the  maximum  entropy  method  (MEM)  in  resolving  closely  spaced 
optical  point  targets  are  compared  using  Monte  Carlo  simulation 
results  for  three  different  examples.  It  is  found  that  the  MEM 
is  very  sensitive  to  the  noise  and  that  the  MLE  performs  better 
than  the  MEM  in  all  examples. 
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INTRODUCTION 


Resolving  closely  spaced  objects  (CSQ)  is  an  important  task 
in  optical  signal  processing.  The  optical  signal  is  often 
subject  to  blurring  by  a  convolutional  function  and  contamination 
by  noise  so  that  the  workable  frequency  bandwidth  is  limited  and 
hence  the  resolution  of  the  restored  signal  is  also  limited. 

Several  nonlinear  processing  techniques  have  been  proposed  and 
employed  with  the  aim  to  increase  the  resolution  power  [1].  By 
incorporating  a  priori  knowledge  that  the  restored  optical 
signal  must  be  positive,  those  techniques  are  generally  able  to 
extrapolate  the  signal  bandwidth.  Among  them,  the  maximum  likeli¬ 
hood  estimator  (MLE)  and  the  maximum  entropy  method  (MEM)  are  two 
common  approaches. 

The  well-known  MLE  has  enjoyed  a  wide  application  in  the  area 
of  estimation.  It  has  been  shown  that  MLE  is  asymptotically  un¬ 
biased  and  efficient  under  certain  general  conditions.  If  the 
noise  is  additive  gaussian  then  this  estimator  becomes  equivalent 
to  the  least-square  method.  The  MEM,  which  was  originally  developed 
to  enhance  the  spectral  resolution  on  short  data  records  [2],  is 
currently  still  a  topic  of  interest.  Besides  spectral  analysis, 

MEM  has  been  adopted  in  a  number  of  other  areas  where  resolution 
is  of  great  concern. 

Early  reports  seemed  to  imply  the  MEM  is  a  method  with  super 
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resolution  power.  However,  recent  studies  have  indicated  that 
its  performance  degrades  significantly  in  the  presence  of  noise 
[3].  In  [4],  it  was  suggested  that  the  MEM  is  superior  to  the  MLE 
in  restoring  optical  impulse  functions  although  no  direct  compari¬ 
son  was  given. 

The  objective  of  this  report  is  to  compare  the  capabilities 
of  MLE  and  MEM  in  resolving  two  optical  CSO's.  Since  an 
analytical  approach  to  this  problem  did  not  appear  feasible,  the 
comparison  was  based  on  the  average  characteristics  of  the  Monte 
Carlo  simulation  results.  Three  examples  which  involved  different 
blurring  functions  were  employed  in  the  simulation  study.  Of 
particular  interest  was  the  third  example  which  dealt  with  the 
CSO  resolution  problem  in  the  environment  of  a  scanning  detector 
and  had  been  studied  in  15]  by  using  MLE.  It  was  found  that 
MLE  could  actually  resolve  the  closely  spaced  optical  targets 
better  than  MEM. 
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II. 


RESTORATION  OF  OPTICAL  IMPULSES  USING  MLE  AND  MEM 


The  problem  of  optical  impulse  restoration  is  illustrated  by 
Fig.  1.  In  many  applications,  the  optical  objects  are  located  so 
far  away  from  the  sensor  that  they  can  be  practically  regarded 
as  point  targets  and  represented  mathematically  by  the  summation 
of  impulse  functions, 

m 

o(x)  =  ^  os-  6(x-6 .  )  n;l°  (1) 

i=l  1  1 

where  m  is  the  number  of  point  targets;  a.  and  6^  are  the  inten¬ 
sity  and  position  of  the  ith  target.  In  optics,  cu  is  physically 
constrained  to  be  positive.  The  observation  is  assumed  to  be 
related  to  the  input  by 

y(x)  =  o(x)  ©g(x)  +  n(x)  (2) 

where  g(x)  is  the  blur  function  and  n(x)  is  an  additive  white 
gaussian  noise  (WGN).  Note  that  the  symbol  (*)  denotes  convolution. 
Given  the  observation,  the  essential  problem  is  to  estiamte  {a^ 
and  {6.}.  For  the  following  discussion,  we  will  further  assume 
that  m=2  (two  targets),  that  J_oa9  (x)  dx=l  (unit-energy  blur 
function)  and  that  only  discrete  samples  (y^,  i=l,  2,...,N}  which 
are  observed  at  {x^,  i=l,  2,...,N}  over  the  spatial  range  [-X,X] 
are  available. 

The  maximum  likelihood  estimator  finds  a.  and  the  estimates 

l  l 

* 

The  WGN  assumption  is  appropriate  when  the  observation  noise  is 
determined  either  by  circuit  thermal  noise  or  by  "shot’1  noise 
with  very  large  intensity. 
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of  and  8^  by  maximixing  the  likelihood  function  which  is  given 
by 


«2.i>  -  ■  „  eXP  |--i-j(V-Ga)T(y-G2)| 

(2"’  “n  '  2on  > 

where  6= ( 6^ , 82 ) T ,  Y= (y 1 »y2 • • • »VN) T  and 


(3) 


G  = 


rgCx^B^  g(x1-82r 


^g(XN'6l)  g(XN~e2)' 


(4  ) 


0  is  the  standard  deviation  of  the  WGN.  It  can  be  shown  that  [8] 


Ak  m  *1  T 

a  =  (G  G)  G  Y 


and  8_  satisfies  the  following  equation. 


(5) 


where 


L(£)  =  max  L (8  ,B0) 
Bi,82 


L(61'62)  =  yTg(gTg)  gTy' 


(6) 


(7) 


By  defining 


N 


K  (6  )  £  5Zg(x.-B.)  y.  i=l,  2 
1  j=l  J  1  2 


N 

P  (8i»B2)  =  ]^g(Xj-B1)  g(Xj-B2) 


we  can  rewrite  (7)  as 


(8) 


(9) 


L{8)  =--1t|<2(B1)  +  k2(B2)  -2p  k(B1)k(B2)| 


(10) 
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To  obtain  the  optimal  solutions  for  a  and  £  using  (5)  and 
((,)  in  conjunction  with  (7)  or  (10)  usually  requires  a  constrained 
nonlinear  optimization  algorithm  which  is  numerically  complicated 
and  computationally  costly.  A  suboptimal  solution  is  available  if 


we  restrict  6^  and  to  be  coincident  with  the  sampling  positions 

/v 

[6] .  In  this  case,  S  can  be  found  by  searching  for  the  global 
maximum  of  (10)  in  a  double  do-loop  fashion  over  { (x^,x.) }  where 

A 

the  indices  are  such  that  l<_i<j£h  and  the  corresponding  a.  and 

a.  must  be  both  positive.  During  the  search,  the  required 
J  1 

i  =  l,2  and  o  can  be  obtained  simply  by  looking  up  the 

precomputed  and  stored  output  sequences  of  the  discrete  matched 
filter  ar.d  the  discrete  autocorrelator  respectively.  If  the  optimal 
solution  is  desired  the  output  of  this  suboptimal  approach  can 
be  fed  into  a  nonlinear  optimization  algorithm  such  as  the 
Quasi -Newton  method  as  the  initial  guess. 

The  maximum  entropy  method  for  restoring  the  optical  impulse 
function  is  shown  in  Fig.  2.  The  MLM,  wnich  is  a  nonparametr ic 
approach,  finds  o(x),  the  restored  signal,  by  maximizing  its  entropy 


K 


o (x) dx 


while  satisfying  the  constraint 


,'x-,  ,  -5*0 

I  o(x)  e  dx. 

J  -X 


(11) 


(12) 
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A 

O(io^)  is  obtained  from  the  windowed  inverse  filter. 

The  choice  of  the  window  width,  of  the  inverse  filter  repre¬ 

sents  a  compromise  between  spatial  resolution  and  signal-to-noise  ratio 
(SNR) .  On  one  hand  the  window  should  be  made  as  wide  as  possible 
in  order  to  provide  higher  spectral  resolution.  On  the  other  hand, 
it  should  be  kept  as  narrow  as  possible  in  order  to  eliminate  the 
noise  effect.  The  idea  of  the  MEM  is  to  extrapolate,  in 
accordance  with  the  principle  of  maximum  entropy,  the  bandwidth 
of  the  signal  at  the  output  of  the  inverse  filter  which  has 
limited  the  signal  bandwidth  to  the  region  where  SNR  is  relatively 
high . 

The  solution  of  (11)  and  (12)  has  been  shown  to  be  given  by 
17) 

(2X)  _i  Yj^+i 

o(x)  - - -X<x<X  (13) 

K  2 

1  +  ^2  Y  expi-jkTr  (2X) (x+X) 
k=l  k  ’ 

with  the  K+l  unknowns  {y^}  determined  as  the  solution  of  the 
K+l  Yule-Walker  linear  equations 


S 


where  wk  =  kTr(2X)-1,  k=C,l,...,K  and 


0'(u)k) 


2R 


e 


0<V 


exp  (-jwkX) 


(15) 


Note  that  the  width  of  the  window  of  the  inverse  filter  is 
effectively  determined  by  the  parameter  K  and  .is  equal  to 
1/2K(2X)  Equation  (14)  can  be  solved  recursively  by  using  the 

Levinson  algorithm  [8],  The  spatial  peaks  of  o(x)  can  be 
identified  by  locating  the  minima  in  the  denominator  of  (13) . 


III.  COMPARISON  VIA  SIMULATIONS 
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Three  examples  were  used  to  compare  the  resolution  capabilities 
of  the  MLE  and  the  MEM.  Here  the  resolution  capability  was  evalu¬ 
ated  only  in  terms  of  parameter  estimation  performance  although 
the  detection  performance  should  also  be  considered.  We  assume 
that  a  priori  knowledge  of  the  number  of  targets  (detection  problem) 
was  available.  Since  we  were  more  concerned  with  the  positions 
of  the  targets,  we  compared  the  two  methods  in  terms  of  the 
performance  (standard  deviation  and  bias)  of  position  estimation. 

For  the  purpose  of  this  comparison,  the  suboptimal  implemen¬ 
tation  of  the  MLE  as  described  in  the  previous  section  was  employed 
in  the  first  two  examples  and  the  optimal  approach  in  the  third 
example.  The  estimation  statistics  were  collected  from  100  Monte 
Carlo  runs.  The  procedure  for  the  MEM  was  more  complicted  because 
the  output  depended  on  the  order  of  the  set  of  linear  equations, 
(14),  and  the  number  of  minima  exhibited  in  the  denominator  of 
(13)  was  not  always  exactly  equal  to  2,  the  number  of  targets. 

For  each  order  of  equations,  the  first  100  Monte  Carlo  runs  which 
displayed  two  or  more  minima  were  used  in  computing  the  statistics. 
For  each  of  those  runs,  the  loc.- tions  of  two  lowest  minima  were 
taken  as  the  position  estimates  of  the  two  targets.  The  best  esti¬ 
mation  performance  (minimum  estimation  variances)  among  all  orders 
of  equations  examined  was  considered  as  the  per f ormance of  the  MEM. 


For  each  of  the  three  examples,  the  separation  A^  between  the 
two  optical  targets  as  well  as  the  SNR  were  varied.  The  SNR  was 
defined  as 


SNR.  =  53  (*.)  )  2  /o 

1  I  j=l  1  3  n 


(16a) 


=  a  .  /  (a  VT)  i=l /  2 


x  n  T  x 


(16b) 


where  <5x  is  the  spatial  sampling  step.  In  all  cases,  SNR^  and 
SNR ^  were  set  equal  and  the  two  targets  were  located  at 
$1 , $2=±AX/2 •  Since  the  configuration  of  the  targets  was 
symmetrical  and  the  blur  functions  used  in  the  examples  were  even 

A  A 

functions,  the  standard  deviations  as  well  as  biases  of  and 
were  expected  to  be  asymptotically  equal.  Therefore  only  the 
average  value  was  used  to  characterize  the  estimation  accuracy. 


Example  1: 


g(x)  =  (no  )  exp[->i(x/o)  ] ,  o  =  10 


N  (sample  size)  =  128 


5x  "  1 


The  blur  function  in  this  example  which  is  similar  to  the  one  used 
in  [4]  is  usually  observed  when  atmospheric  turbulence  is 
significant.  The  associated  autocorrelation  function  p(i) 
and  Fourier  transform  G(u>)  are  given  by 


'  i".’  ■'  HTFWPH*  wf)if  WTEHf '  iwtww  WTTWw,>,‘ . . .  ' '  ~,pr 


(18) 


p  (t  ) 

GU) 


exp  -  [(-*-) 2] 

2  **  2  2 
( 4 7T o  )  exp(-»aO  uj  ) 


(19) 


The  performances  of  the  MLE  and  the  MEM  are  compared  as  shown 

in  Fig.  3.  The  standard  deviation  o  and  the  bias  b  of  the 

xx 

posi tion  estimation  are  plotted  against  the  target  separation 
A  for  SNR  =  20  and  10.  For  either  of  these  two  methods,  it  can 

X 

be  observed  that  the  performance  degrades  when  becomes  smaller 
or  when  SNR  is  reduced.  However  when  compared  with  each  other, 
it  is  obvious  that  the  MLE  is  better  than  the  MEM  in  the  sense 
of  smaller  o  and  b  for  almost  all  combinations  of  A  and  SNR. 

XX  X 

From  Fig.  3,  it  can  also  be  seen  that  the  two  targets  behave 
like  isolated  targets  for  larger  A  (e.g.,  >  3o)  where  o  and  b„ 
remain  almost  constant.  If  the  effect  of  £  (equal  to  1)  on  the 

X 

estimation  accuracy  is  taken  into  consideration,  it  is  acceptable 

to  say,  based  on  data  from  Fig.  3,  that  in  this  separation  range 

either  MEM  or  MLE  is  almost  unbiased  (b  <6  )  and  their  standard 

x—  x 

deviations  are  close  to  the  square  root  of  the  Cramer-Rao  bound* 
for  a  single  target  which  is  equal  to  .707  for  SNR=20  and  1.414 
for  SNR=  10. 


* 

The  Cramer-Rao  bound  (CRB)  is  the  lower  bound  on  the  variance  of 
an  unbiased  estimator.  For  a  single  target,  the  CRB  of  the  position  estimate 

is  equal  to  1/(SNR»B)^  where  B  defined  as  B2= J'  w2G2(o)dw/  y*°°  G2(w)dio  is 

the  so-called  root-mean-square  bandwidtfi.  In  example”0!,  B-.0707. 
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v° 

Fig.  3(a).  Performance  comparison  of  MLE  and  MEM,  example  1. 
Estimatimation  bias,  bx,  is  plotted  as  a  function  of  target 
separation.  Ax;  Ax  is  normalized  with  c  of  g(x). 
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Example  2: 


'g(x)  =  ( 3/2 )  **  sin2  (ttx)  /  (ttx)  2  ,  x=6/(X/D)  (20) 

N=128 

I  6  =.  1 

i  x 


Here  g(x)  is  the  blur  function  due  to  a  slit  aperture  with 
length  equal  to  D.  The  classical  Rayleigh  resolving  power  in 
angle  (0)  for  this  type  of  aperture  is  equal  to  X/D  where  X  is 
the  wavelength  of  the  incident  light.  Tne  associated  p(t)  and 
G(uj)  are  given  in  the  following: 

p(T)  =  iTTT^SinCjjTTj  ( 

2  (ITT  ) 


G  (id)  H 


I  (D  I  £2  IT 


otherwise 


(22) 


Note  that  this  blur  function  is  bandliraited. 

Figure  4  shows  the  comparison  oi  the  MLE  and  the  MEM  using 
this  example  with  SNR=20.  Obviously,  the  MLE  is  much  better  than 
the  MEM  in  estimating  the  positions  of  the  optical  CSO's.  Note 
that  the  hump  of  the  MLE  ox  curve  is  due  to  the  first  sidelobe 
of  g (x) . 
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Fig.  4(a).  Performance  comparison  of  MLE  and  MEM,  example  2. 
Estimation  bias,  bx,  is  plotted  as  a  function  of  target  sepa¬ 
ration,  A  . 


Fig.  4(b).  Performance  comparison  of  and  MEM,  example  2 

Estimation  standard  deviation,  a  ,  is  ^otted  as  a  function 
of  target  separation  A  .  x 


Example  3: 


This  example  deals  with  the  CSO  resolution  in  the 
environment  of  a  scanning  detector.  The  problem  has  been  treated 
in  [5]  using  the  MLE .  A  brief  description  is  as  follows.  A 
detector  scans  through  the  focal  plane  of  an  optical  sensor  on 
which  an  image  due  to  a  pair  of  CSO ' s  in  the  f ield-of-vi ew  of  the 
sensor  has  been  produced.  The  spatial  structure  of  the  image  is 
thus  converted  into  a  temporal  signal.  Without  loss  of  generality, 
a  unity  scanning  rate  can  be  assumed  so  that  the  spatial  and  time 
coordinates  are  equivalent.  It  is  also  assumed  that  the  two  CSO ’ s 
are  lying  in  the  scan  direction  (e.g.,  x-axis).  The  blur  function 
g(x)  appearing  in  the  signal  model,  Eq.  (2),  is  now  given  by  the 
convolution  of  the  point  spread  function  p(x,y)  and  the  detector 
response  function  d(x,y).  Here  in  this  example,  p(x,y)  is  due  to 
an  annular  aperture  with  diameter  equal  to  2  A/D  (A/D  is  the  optical 
diffraction  limit)  and  obscuration  coefficient  equal  to  50%,  and 
d(x,y)  is  assumed  to  be  uniform  over  the  detector  surface  whose 
dimensions  are  2  A/D  in  scan  direction  and  6  A/D  in  cross-scan 
direction . 

In  this  example,  no  closed  form  expressions  can  be  obtained 
for  g(x)  and  G(w)  and  therefore  they  are  computed  numerically. 
Sixty-four  samples  taken  at  .2  A/D  interval  are  used. 

In  Fig.  5,  the  root-mean-square  (RMS)  errors,  e^,  of  the 
position  estimation  using  MLE  and  MEM  are  plotted  as  functions  of 
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the  target  separation.  Both  variables  are  expressed  in  terras 

of  the  detector  width  in  the  scan  direction.  Note  that 
2  2  15 

=  (b^  +  0  )  .  Again,  it  is  obvious  that  the  MLS  can  resolve  the 
CSO's  much  better  than  the  MEM  in  this  example. 


■i 
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IV.  CONCLUSION 

Based  on  simulation  results  with  three  diverse  examples, 
it  can  be  concluded  that  the  maximum  entropy  method,  as  compared 
to  the  maximum  likelihood  estimator,  is  not  an  outstanding  approach 
for  increasing  the  resolution  of  two  closely  spaced  optical  point 
targets  particularly  in  the  presence  of  noise.  The  difficulty 
with  the  MEM  lies  in  the  fact  that  it  is  derived  without  considering 
the  measurement  noise  and  it  can  only  depend  on  the  windowed  inverse 
filter  to  reduce  the  noise  effect.  Although  some  modifications 
including  the  noise  compensation  scheme  [9]  and  another  maximum 
entropy  formulation  [10]  have  been  proposed  to  deal  with  the  noise, 
the  results  are  still  not  very  convincing.  Besides,  the  simplicity 
of  the  MEM  has  been  lost  in  these  new  approaches. 


-J 
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